Measuring Testing Complexity

Since the birth of Software Industry there is a special interest in measuring the effort in terms of time and cost that a task requires. Nowadays, since applications are essential for Industry, the software developers need to measure different kind of aspects: quality, cost, reliability, etc. Complexity is another important aspect in which software engineers are interested. But, what does we mean with “complexity” of a software piece? Basili [1] defines complexity as a measure of the resources used by a system while interacting with a piece of software to perform a given task. If the interacting system is a computer, then complexity is defined as the execution time and storage required to perform the computation described by the program. If the interacting system is a programmer then complexity is defined as the difficulty of performing tasks such as coding, debugging, testing or refactoring. In this abstract we propose a new meaning for complexity: the difficulty of testing a program. Such a measure of complexity, which we call Testing Complexity, could be useful to decide which is the best way of generating a test suite for the piece of work. It could be useful, for example, to decide the parameters of an evolutionary test data generator [3] prior to its execution. Our new complexity measure predicts the behaviour of an automatic test data generator based on random testing [2]. The measure is based on a Markov model of the program that is used for estimating the probability of traversing the branches of the program. In order to confirm the quality of our new complexity measure, we performed an empirical study using a benchmark of 1800 test programs automatically generated.